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PROCEDURE. USING A 7-FOINT SCALE, EACH PARTICIPANT RATED 
HIMSELF ON (1) SFEAKIfJG VOICE, (2) MANNERISMS, (3) KWriJLECGE 
CF SUBJECT,. (4) HIS ENTHUSIASM, (5) CLASS ENTHUSIASM, (6) 
DIGRESSICWS, (7) C^iGANIZATlC'N AND FREFARATICW, (6) USE CF 
ANALOGIES, EXAMPLES, AND ILLUSTRATIONS, (9) HANDLING CF 
OUESTI*jNS, and (10) GENERAL CLASS ATWXFHERE. HE THEN 
RECORDED TWO CLASS SCSSICNS DURING A 2-WEEK PER ICC, AND 
FXLCWED HIS REVIEW OF THE TAPES BY ANOTHER SELF-RATING. 
ALTH-OUGH fO SIGNIFICANT DIFFERENCES WERE FOUND IN THE TWO 
RATINGS, AFTER LISTENING TO THE TAPES CF THEIR CLASSES, FIVE 
TEACHERS RATED THEMSELVES MiORE FA\A:«RABLY AND SIX PLACED 
THEMSELVES LOWER «jN THE SCALES. THUS, THERE IS EVIDENCE THAT 
OVER HALF CF THE PARTICIPANTS WERE SENSITIVE TO THE 
INFORMATICS C6TAINED FRi:«M THE TAPES. THE AUTHOR CCWCLUDED 
THAT, ALTHICCj: LACK «0F EXTERNAL CRITERIA MAKES THIS PRiXEDURE 
UNSUITABLE FOR MERIT RATING FURF«0SES *0R FOR COMFARIS:s CF CSE 
INSTRUCT*XI WITH ANOTHER, THE fCTHrO HAS PROMISE FOR THE 
INDIVIDUAL TEACHER’S SELF-IMPROVEMENT ACTIVITIES. (WO) 
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As the population explosion continues to be felt 
In the colleges and universities, the expansion of present 
facilities and the planning of new ones have become part of 
the educational scene on campuses all across the country. 

As new buildings and college® spring up as If by 
magic, legislators and the public may point with pride at 
the tangible evidence of their concern about the future of 
higher education In America. But, the creation of each 
new campus, the completion of each new building, and even 
the establishunent of each new curriculum are not without 
their drawbacks. For the problem of staffing each facility 
Is no longer a bridge to be crossed In the future. It Is 
linmlnent, and the available supply of experienced faculty 
Is limited. In fact, at present, most Institutions have 
one or more unfilled vacancies on their staffs because 
of the lack of qualified candidates. Further, data from 
graduate schools and other sources of potential faculty 
indicat ■■ that the snortage will be^come increasingly 
severe into the 1970' s. Consequently, department heads, 
deans, and presidents have intensified their efforts to 
not only acquire but retain competent qualified faculty 
for t h i r ins t i t ut ior.s . 
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To this end, armed with daoa concerning sol. ring 
enrollment, teacher shortage, and current competitive 
salaries, most presidents have been able to slowly but 
surely extract increasing amounts of money from legislators 
or trustees for faculty salary raises. This has been a 
continuing process and promises to be more so in the fore- 
seeable future. However, this largesse is not without its 
attached strings. Insistence has been made that raises 
be made contingent upon merit --and not given across the 
board. The dean, unenviable creature that he is, when 
confronted by the president on this point, agrees whole- 
heartedly and with rapidity, because he is under pressure to 
cover classes and the projected salary increase may spell 
the difference between a few restful nights and the nightmare 
of searching for two or three newly vacated positions. Thus 
assurances are given that only the meritorious shall be 
given Increments. 

Of course this entire situation would be rather 
amusing if it weren*t so tragic — and so real! Because for 
years the brightest and best trained men have gone into 
Industry and more recently into research as the salary 
structure there has disproportionately increased. Now, 

ysars of selective starvation, the college personnel 
are asked to reward only the outstanding, and miracile 
dictu, there wtill remain in colleges, some who are out- 
standing. This is either a tribute to the dedication and 
altruism of the teacher, or a monument to his colossal 






ERIC 

Mj'lliiHililRIffTlTliilU 













119 







~^T.* 






£ _r-*> ' 



1^* 















•r>-r 



er|c 






- * >» 'V ^ . • 






loa 



* ot"\^ ^ ‘‘TP 

~ ^ V ^ ' CA'.4. C • 



Be that as it may, however, most components of the 
educational system are happy. The faculty has i' s ral es, 
U\e public has its newspaper account of how merit raises 
only are to be given, the legislature or trustee body has 
the premise of the board of control or regents, the regents 
have the proi.:ise of the presidents, the presidents have 
the as3urar.es of their deans, and finally, the deans have 
their classes covered and ulcers palliated temporarlly-- 



but only temporarily. 

But soon, as swiftly the dawn the night, comes the day 
of reckoning. Someone, typically the dean, must show evi- 
dence of good faith in attempting to either devise or dis- 
cover a discriminating m.erlt scale, or else be prepared 
to defend his conclusion that all his faculty are Indeed 
merlto.rlous, thus deserving of an Increment across the 
board. Since we usually end up with the latter approach 
anyvjay, the scope of this paper is limited to the first 
alternative , 



The most frequent and most discussed approach to 
merit rating is the construction of a scale or vjhlch 

would reveal good or desirable pedagogical practices. Most 
have been oriented around the concept ci good teaching. 
Rating scales or other evaluative devices, filled out by 
students, former students, peers, or superordinates, have 
been most prevalent. Howf;ver, tne many reliability and 
validity problems, including definitions and criterion 






measures, in addition to the attendant faculty morale 
disturbances, will not be dealt with since they are well 
known to all administrators. Generally, this approach has 
been attempted, discarded, attempted again with a new 
twist, and subsequently discarded again almost with a monot 
onous regularity. The question of whether this technique 
Is batter than nothing remains moot in the eyes of many 
deans . 

Another approach, less formal and less Infoimiatlve 
also, is class visitation. Often this Is done by the dean, 
department head, a seasoned and respected teacher;, or a 
peer. Problems of open faculty resentm.ent and sampling 
bias are of course manifest, but of more pragmatic concern 
to the dean is the inordinate amount of tline which must be 
devoted by the visiting evaluators to the visitation 
schedule. Since, If any type of systematic and comparative 
information is to be derived, a definite plan and adequate 
coverage must be obtained, but, because of the time factor 
that this arrangement necessarily entlals, most deans 
have not relied heavily on this method of assessment. 

A third approach has been the follow-up of former 
students in subsequent relevant courses. At the Junior 
college this has been particularly difficult because of the 
necessary cooperation of the many and diverse senior 
institutions to wh-»ch transfers have been sent. In addi- 
tion, because of the time delay Involved, faculty are 
frequently gone by the time such data are available. Senloi 
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instit’itions have somewhat the same problems with reference 
to graduate schools, but within their four-year confines, 
institutional research programs could be most revealing 
in this area. To my knowledge, there have been no 
published data on this to date. This method has not been 
used to its fullest extent. 

A fourth approach to the assessment of faculty merit, 
and one heavily leaned upon, is that of talks, lectures, 
symposia, or publications. But, that there is a consider- 
able difference between quotidian class delivery and special 
discursive forums or written articles, is too obvious to 
dvjell upon. The chief positive values of this approach are 
that it is quantifiable, gets good press, and, of co\irse, 
is easy to do. The chief drawback is that it is not relevant. 

A fifth approach is rather — and better said — no ap- 
proach at all, but a melange of rumor, student reports, 
hr-i rsay, and the typical scuttlebutt pervading a college 
campus. As a feedback mechanism, this approach has heuris- 
tic value, but as a dlscriminandum for merit it is not only 
useless, but can do a great deal to destroy faculty morale, 
and the respect of the dean by the faculty. 

A sixth approach, and the one on which this paper 
reports, is that which I term the auto-critique method. The 
basic ingredient is that the Instructor evaluates himself. 

The chief disadvantages are that the method is essentially 
self contained, is not readily relatable to an external 
criterion, and is not amended to tangible control or 



manipulation by the administration. But, since upon scru- 
tiny, the other approaches have these tvro disadvantages as 
well, as has been pointed out previously, the advantages 
of this method offer themselves as worthy of the considera- 
tion of a dean who :Ls attempting to discharge his obliga- 
tion to assess and \ipgrade the instructional level of hls^ 
institution. Amond these advantages are: 

1. Evidencing Interestest in the teaching process 
Itself by the administration, 

2. Indicating confidence by the administration in 
the faculty* s ability to evaluate themselves as professional 
and make self indicated Improvements, 

3. Giving the faculty a workable and frequently 
interesting method of whereby they may Improve themselves, 

4. Preservation of anonymity by faculty, thus fore- 
stalling feelings of ”blg brother" watching, 

5. Establishing essentially a self operating and 
perpetuating system not calling for a great amount of time, 

6. Placing of the dean in the position of being 
called in for aid by a motivated faculty member, rather than 
being looked upon as an Intruder with unwanted advice, 

7. Providing specific and concrete examples (pre- 
served on tape) of problem areas which can be referred to 
upon replay, and not having to rely on notes or faulty 
memory . 



Method 

The materials used in the current investigation were 
a rating sheet and a tape recorder with two on-hour tapes. 
Nineteen members of the 26 full-time faculty volunteered 
to participate. The rating scale was constructed after 
consultation with various experienced former faculty col- 
leagues of the author at Florida Gtate University, and a 
review of the items appearing in various published teacher 









rating scales. Ten different areas were covered with the 
categories of "verbal delivery and "enthusiasm" being 
further subdivided. A seven point rating., scale was then 
Imposed on each category with the directions asking the 
instructor to rate himself from low (l) to high (7) 
as to his performance in each category. The final category 
items were: 

1. Speaking voice 

2. Mannerisms or pleonasms 

3. Knowledge of subject matter 

4. Enthusiasm - self 

5. Enthusiasm - engendered in class 

6. Digressions 

7. Organization and preparation 

8. Use of analogies, examples, illustrations, etc. 

9. Handling of questions 

10. General atmosphere created. 

Anonymity was preserved, although for statistical 
treatment each Instructor was asked to assign himself an 

r 

easily remembered code name or number. After the form 
was filled out, it was then turned into the dean's office. 
Two one-hour tapes and a recorder were then made available 
to each Instructor. His instructions were to start the 
tape at the beginning of the class hours, and turn it 
off at the end. This was to be done for two class periods 
within a two-week period. After both recordings were made, 
the instructor was to pick up another copy of the rating 
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sheet, listen to both tapes, and then refill out the rating 
sheet which was also turned Into the dean’s office. Thus 
a comparison could be made of the instructor’s original 
and subsequent ratings of himself. Tapes were reused at 
will by the Instructor. Instructors were encouraged to 
discuss specific problems emerging from the sessions with 
an experienced teacher, the department heads, or the dean. 

The forms were then analyzed for information. 

Results 

Since the ultimate purpose and hence the real value, 
of this auto-crltlque method was to have each Instructor 
devote his professbnal skills, talent, time, and energy 
to a crltlccil self assessment, statistical treatment of 
group data was considered of subordinate value. Neverthe- 
less, cursory analyses were made and certain conclusions 
reached. These data will be presented below. Comments 
elicited from the faculty reflecting the self assessment 
component of the method are also listed. Thus, both quan- 
titative and qualitative aspects are noted. 

The data presented in Table 1 are by columas: 

Column 1. The pooled mean score (N«19) hy category 
on the first or Before administration of the scale. 

Column 2. The same for the 2 or After . 

Column 3. Directionality of change. 

Column 4. Pooled rank of each category In comparison 
with the other categories on the Before. 

Column 5. Same on After . 

Below the table you will note the grand mean for the 
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and a Spearman rank 



Before and the After administrations, 
order correlation coefficient. 

The statistical analysis included; 

1. An analysis of variance and F test. 

2. White *s Rank Order test of signiflcarxce. 

3. Spearman Rank Order correlation method and test 
of significance for sampling variance. 

As can be readily inferred, no statistically signifi- 
cant differences between the Before and After administra- 
tions of the questionnaire were found using either the 
parametric analysis of variance or the non parametric Rank 
Test. Further, the coefficient of correlation between the 
two adminlstratl)ns of the scale was .37^ significant beyond 
the .01 level, hence indicative of a quite high and re- 
liable relationship. The conclusion that there was not an 
overall change in self rating on this seven point scale 
as a fijnction of the listening to the taped sessions is 
readily made. However, since on a rating scale in which 
there is a compressed continuum, i.e., seven points plus a 
consldei’able halo effect operating — note the skewing to- 
ward the high rather than middle or low end of the scale — 
a factor suggesting that the faculty think highly of them- 
selves on both administrations, which is not unrealistic 
since the college thinks highly of them also, and has re- 
inforced this view both with verbal praise and behavioral 



acts. On such a rating scale, it is not unexpected that 
numerical data are not significant. However, much informa- 
available from inspective anajysir. of tne pattern 
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•s nade. Along thosi lineo, some Inter-ectlng 
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points emerge. They are: 

1. V.’jLlest range = Category 5- (Enth- Eng-Class) 

2. Constricted = categor'y 9, (Handling Questions) 

Highest rank Befoi^=Category 3, (K of SugJ. Matter) 

Highest rank After =Category 3 , (K of Subj .Matter) 

Lowest rank Before =Category 2, (Mannerisms or 
Pleonasms) 

Lov^est rank After =Category 5^ (Enthus. -Class) 

7. Of soeclai interest is Category 6, (Digressions), 

vjhich shifted from 8th to 6th ranking. This ^ 

the faculty oerceive themselves as digressing less after 
listening to* their tapes, than they originally thought. 

The data in Table 2 indicate the number of faculty 
(total K=19) vjho, as a function of listening to the tape, 

shifted : 

1. up ( =, thus giving a more favorable rating to 
themselves) , 

2, down (-, thus giving a more unfavorable rating 
to themselves), 



3. no chamge. 

It is to be noted that five of the ten categories 
shovjed a directional change by a majority (<L0/l9) of the 
raters. These categories were: 



1. 


Speaking Voice, 




7. 


Organization and Preparation, 




2. 


Mannerisms or Pleonasms, 




5. 


Enthusiasm Engendered in Class, 


and 


• 

o 

I — 1 


General Atmosphere Created. 




Table 2 also reveals the ranking of 


each category 
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Table 2 Directional change by frequency for each category, (N«19)> and rank of susceptibility 
to change of each category. 
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from r.ost susceptible to least susceptible to change of 
rating by the faculty as a function of listening to the 
taped lectures. Categoiy 1, (Speaking Voice) ranked first 
in susceptibility to change, while category 3 (Knowledge 
of Subject Matter) was low or least susceptible. Inter- 
vlevjs with experienced faculty revealed that they could not 
acclimate themselves to the soiond of their own voice regard- 
less of past auditory exposure. Of Interest are the values 
of category 6 (Digressions), and category 10 (General At- 
mosphere) with ranks of 7-5 and 3 respectively. This may 
indicate that faculty are more fearful of digressing than 
the facts might warrant, and that atmosphere effects con- 
tinue to be a rather evanescent quality. 

Table 3 

Number of Faculty who changed ratings preponderantly 

(>5) upward = 5 

Number of Faculty who changed ratings preponderantly 

(>p) dovmward - 6 

Namber of Faculty vjho had nc preponderant shift = 8 

TOTAL 19 

The data in Table 3 Indicate the number of faculty who 
changed their rating of themselves on the second adminis- 
tration, and the directionality of the change, on five 
or more of the ten categories . Of the nineteen faculty 
Involved, five rated themselves more favorably the second 
administration, six less favorably, and eight did not change 
their ratings on five or more categories. Thus, more than 
half of the faculty (ll/l9) appeared sensitive to the Inform- 
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atlon obtalnsd from tho tape. It Is In this sense, then, that 
the auto-crltlque method seems of value. 

In the more qualitative or second facet of the data 
analysis, som.e comments made by the faculty respondents 
are listed below. 

1. The day recorded happened to be a good day! Not 
every day Is like that. 

2. After hearing recording of lecture — hard to de- 
termine enthusiasm of students — difficult to pick up on 
tape* 

3. Lecture was very boring, I think! 

4. I know It, although I often can*t articulate It well. 

5. The fact that this was a "tape’* som.ewhat cut dovm 
on the class response — perhaps I was more formal — but I 
found this whole thing enlightening! 

6. Good questions and enthusiasm following tape 
(2nd period) . 

7. Pace could be speeded up. 

8. I believe my major fault Is In delivery— lack of 
Inflection and too many pauses while trying to phrase 
thoughts. I think this is offset by Inspiring Interesu 
through relating subject matter to everyday life. 

9. Ply delivery is too slovj — too many pauses also too 
many repetitions. 

10. feel that students were more Interested In re- 
corder than they were my lecture. 

11. Too many long pauses— repeating words--" It is 
clear", "rather obvious", *’we know", "so we see . 

Again, these comments seem Illustrative of the point 
that this technique can provide feedback to the interested 
Instructor. 



Conclusions 



The prime conclusion reached by the invesuigator is t:i)t 
the auto-critlque method is not a u-efui instrument in 
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e^'-aiuatlng instruction for a merit criterion purpose. Be- 
cause of lack of external (to the individual Instructor* s 
ov;n frame of reference) criteria, it does not seem to have 
objective value for inter-instructor or instructor-criterion 
comparisons. A corollary, if secondary, conclusion, however, 
is that the auto-critique method has great value for in- 
structional Improvement on the part of the average as well 
as highly motivated Instructor. In addition to the anony- 
mity and self pacing features as Indicated in the intro- 
ductory passages of this paper, the results, both quantitative 
and qualitative, suggest that informational feed’ ack of a 
differential nature is provided the instructor. Thus, the auto 
critique method, as it relates to instructional Improvement, 
seems vjorthy of continuation and Columbus College has incorpor- 
ated it into the general orientation program for new faculty 
during their first year on campus. It is hoped that this 
miethodj with additional experimentation, preferably with 
video tape, will ultimately produce increasingly better 
teachers . 



